Web Survey Bibliography
A goal of the survey interview is to collect reliable and valid data. Achieving this goal is often difficult because respondents may not understand what is being asked of them. In traditional interviewer-administered survey modes, interviewers can pick up on signs of confusion and difficulty answering a question from the respondent’s speech patterns, expressions, or response times. In self-administered surveys, however, identifying confused respondents has previously not been possible. The introduction of Web surveys provides an interactive environment with a vast amount of data that researchers can collect in real time. Using these data, it may be possible to determine when respondents are having difficulty answering a question, much like in an interviewer-administered survey. Using Web browsing and education research as a basis, this paper identifies 11 unique movements that respondents make with the mouse cursor while answering survey questions. Through an exploratory analysis, we hypothesized which of these movements are related to difficulty answering survey questions. Then, using scenarios to manipulate question difficulty and asking participants to rate the difficulty of each question, we were able to test our hypotheses to determine which movements are related to difficulty and which are general movements people make when interacting with a computer. Finally, this paper proposes a model that can be used to predict, in real time, when a respondent is having difficulty answering a survey question. We find that not only are certain mouse movements highly predictive of difficulty, but they are more predictive than response times, which have been used to predict difficulty in the past. This information can be used to provide real-time help to confused participants or it can act as a diagnostic tool to identify confusing questions.
Conference Homepage (abstract)
Web survey bibliography (95)
- Virtual reality meets sensory research; 2017; Depoortere, L.
- Methods for Evaluating Respondent Attrition in Web-Based Surveys; 2016; Hochheimer, C. J.; Sabo, R. T.; Krist, A. H.; Day, T.; Cyrus, J.; Woolf, S. H.
- Exploration of Methods for Blending Unconventional Samples with Traditional Probability Samples; 2016; Gellar, J.; Zhou, H.; D.; Sinclair, M. D.
- Ratio of Vector Lengths as an Indicator of Sample Representativeness ; 2016; Shin, H. C.
- Online and Social Media Data As an Imperfect Continuous Panel Survey; 2016; Diaz, F.; Garmon, F.; Hofman, J. K.; Kiciman, E.; Rothschild, D.
- Validating self-reported mobile phone use in adults using a newly developed smartphone application; 2015; Goedhart, G., Kromhout, H., Wiart, J., Vermeulen, R.
- Innovative Uses of Paradata Across Diverse Contexts ; 2015; Cheung, G.; Pennell, B.-E.
- Build your own social network laboratory with Social Lab: a tool for research in social media; 2014; Garaizar, P., Reips, U.-D.
- Picking up the Bread Crumbs: Holistic Insights from Social Media; 2014; Souda, P.
- Survey optimisation considerations for Android, Apple and Windows 8 mobile devices; 2013; Owen, R.
- Second Life as a Survey Lab: Exploring the Randomized Response Technique in a Virtual Setting; 2013; Richards, A., Dean, E.
- Virtual Cognitive Interviewing Using Skype and Second Life; 2013; Dean, E., Head, B., Swicegood, J. E.
- Assessing Nonresponse Bias in the Green Technologies and Practices Survey; 2013; Meekins, B., Sverchkov, M., Stang, S.
- Why Big Data is a Small Idea…and Why You Shouldn’t Worry So Much; 2013; Needel, S.
- Doing real time research: Opportunities and challenges; 2013; Back, L., Lury, C., Zimmer, R.
- Digital technology and data collection; 2013; Henriksen, B., Jewitt, C., Price, S., Sakr, M.
- Effects of Self-Awareness on Disclosure During Skype Survey Interviews; 2013; Feuer, S., Schober, M. F.
- Cognitive Interviewing in Online Modes: a Comparison of Data Collected in Second Life and Skype; 2013; Swicegood, J. E., Head, B., Dean, E., Keating, M.
- Effects of Displaying Videos on Measurement in a Web Survey; 2013; Mendelson, J., Gibson, J. L., Romano Bergstrom, J. C.
- Classifying Mouse Movements to Predict Respondent Difficulty; 2013; Horwitz, R.
- Are You Seeing What I am Seeing? Exploring Response Option Visual Design Effects With Eye-Tracking; 2013; Libman, A., Smyth, J. D., Olson, K.
- Survey Reminder Method Experiment: An Examination of Cost Efficiency and Reminder Mode Salience in the...; 2013; Anderson, M., Rogers, B., CyBulski, K., Hall, J. W., Alderks, C. E., Milazzo-Sayre, L.
- Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds; 2013; Hasler, B. S., Tuchman, P., Friedman, D.
- From mixed-mode to multiple devices. Web surveys, smartphone surveys and apps: has the respondent gone...; 2013; Callegaro, M.
- Internet-Mediated Technologies and Mixed Methods Research; Problems and Prospects; 2012; Hesse-Biber, S.; Griffin, A. J.
- An Introduction to Using Video for Research; 2012; Jewitt, C.
- Online Surveys Aren't Just for Computers Anymore! Exploring Potential Mode Effects between Smartphone...; 2012; Buskirk, T. D., Andrus, C.
- Smartphone Apps and User Engagement: Collecting Data in the Digital Era; 2012; Link, M. W.
- Specific mixed-mode methodology to reach sensory disabled people in quantitative surveys; 2012; Fontaine, S.
- Facing The Future Webcams as a survey tool in China; 2012; Gordon, A., Llewellyn, T., Gu, E.
- Comfortable in the new medium: How online qual can benefit from our share-happy culture ; 2012; Rubenstein, P.
- Using Collaborative Web Technology to Construct the Health Information National Trends Survey; 2012; Moser, R. P., Beckjord, E. B., Finney Rutten, L. J., Blake, K., Hesse, B. W.
- The Representativity of Web Surveys of the General Population compared to Traditional Modes and Mixed...; 2012; Klausch, L. T., Schouten, B., Hox, J.
- Time use data collection using Smartphones: Results of a pilot study among experienced and inexperienced...; 2012; Scherpenzeel, A., Sonck, N., Fernee, H., Morren, Me.
- Using Webinar Polls to Collect Online Survey Data: The Case of a Behavioral Finance Problem; 2012; Sahu, C.
- The Game Experiments: Researching how gaming techniques can be used to improve the quality of feedback...; 2011; Sleep, D., Puleston, J.
- The benefits and constraints of e-mail interviews and discussions as methods of accessing valid data; 2011; Roberts, An.
- Facial imaging: The new face of online survey research; 2011; Gordon, A., McCallum, D., Sorci, M., Llewellyn, T.
- On Affordances and Technological Intersubjectivity; 2011; Vatrapu, R.
- Building online communities; 2011; Mlačić, B., Milas, G., Mikloušić, I.
- Eye Tracking in testing questionnaires: What’s the added value?; 2011; Tries, S.
- Video enhanced web survey; 2011; Fuchs, M., Kunz, T., Gebhard, F.
- Engagement, Consistency, Reach – why the Technology Landscape Precludes All Three; 2011; Johnson, A., Rolfe, G.
- Twitter mood predicts the stock market.; 2011; Bollen, J., Mao, H., Zeng, X.-J.
- Web based CATI on Amazon Elastic Compute Cloud and VirtualBox using queXS; 2011; Zammit, A.
- Web/Cloud Based CATI Using queXS; 2011; Zammit, A.
- Partnership-Driven Resources to Improve and Enhance Research (PRIMER): A Survey of Community-Engaged...; 2011; Dolor, R. J., Greene, S. M., Thompson, E., Baldwin, L.-M., Neale, A. V.
- Weaving the Web into Personal Communication Networks: A Mobile Phone Based Study of Smartphone Users; 2011; Kobayashi, T., Boase, J.
- Different functioning of rating scale formats – results from psychometric and physiological experiments...; 2011; Koller, M., Salzberger, T.
- Measurement invariance in training evaluation: Old question, new context; 2011; P., Gissel, A., Stoughton, J. W., Whelan, T. J.Clark, A. P.